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ABSTRACT 

The role of evaluation in a working model for the 
diffusion of educational innovations is the central topic of this 
paper. The working model is a product of the Trent Valley centra 
(TVC) , a "regional laboratory" type extension center of the Ontario 
Institute for Studies in Education. The first objective of the 
program is to generate the working model for the diffusion of 
educational innovations, both process and product, and the second 
objective is to implement the emerging model in a school system that 
is characterized by the absence of markedly atypical features. The 
major components of the model include: a climate for change; 
academic-practitioner interaction; roles for evaluation; program 
development strategies; interschool cooperation; county wide 
communication networks; and teacher responsibility for change. Stages 
in the model include: agreement to begin; establishment of an 
organization; selection of problems and goals; study of available 
solutions; pilot trials; adopt, adapt, reject decisions; and field 
trial. The major contribution to new knowledge that this model makes 
has to do with the ways in which the components interact to produce 
validated educational change. (Author/DB) 
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The role of evaluation In a working model for the diffusion of 
educational Innovations Is the central topic of this paper. The working 
model In question Is a product of the Trent Valley Centre (TVC) which Is 
a "regional laboratory" type extension centre of the Ontario Institute for 
Studies In Education. Since the time of Its establishment In September 1969 
the TVC focussed most of Its resources on a program that alms to Identify 
general procedures for Improving student performance on school objectives 
through the application of educational research manpower and Information. 

The first objective of the program Is to generate the working model for 
the diffusion of educational Innovations, both process and product, and 
the second objective Is to Implement the emerging model In a school system 
that Is characterized by the absence of markedly atyp.cal features. The 
third and fourth objectives of the program, concerning field trials of 
the model, and comparisons of the model with competing models, are not In 
an operational sta./e and hence remain beyond the scope of the present 
paper. 



The Trent Valley Centre Is located In Peterborough, a city of 
nearly 60,000 population approximately ninety miles northeast of Toronto, 
Ontario. During the Initial year of operation of the TVC there were five 
schools participating directly In the TVC Program. The number of par- 
ticipating schools expanded to seventeen during the second year, and now 
In the third year there are approximately thirty schools Involved In the 
- TVC Program. A full report of the second year v s activities Is available 
In "Educational Change and Innovation} A Case Study" (Russell, Leithwood 
and Baxter, In press). 
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Rationa l fo r the Trent Valiev Centre Program 

The questions why evaluate, why Innovate, and why plan the diffusion 
of Innovations In education, have answers which seem self-evident, and yet 
there are many Issues raised by these questions that remain unresolved In 
most school settings. School Innovations are typically Introduced to better 
achieve existing school goals or objectives. Under such circumstances. It 
Is necessary to conduct a careful evaluation to determine the exact extent 
of the advantage of the Innovation over the previous program. It Is 
reasonable also to Introduce Innovations for the purpose of achieving new 
and more desirable school goals. Again evaluation Is an essential component 
In the total scheme, because In the absence of evaluation data It Is possible 
that the new goals ore achieved only superficially by the Innovative program. 
Another reasonable purpose for Innovating Is to Increase the proportion of 
students who achieve objectives that are previously achieved by too small 
a proportion of students. In each of the above cases of gains, there Is the 
possibility of a cost factor which may be prohibitive. The Issue of cost 
raises the possibility of Innovations that are designed primarily for the 
purpose of reducing the dollar or manpower costs. In such cases even a 
zero gain In student performance may be acceptable. In the event that any 
of the above types of benefits to society can be made or are made In a school 
setting, extension of the benefits over a wider range of schools and students 
Is warranted. The TVC study assumes that planned diffusion can be more 
effective than either unplanned diffusion activities or standard dissemination 
procedures. This Is a testable hypothesis which may be jccepted or rejected 
on the basis of empirical verification at a later date. 
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Thor* aro very basic questions that remain unanswered In the 
rationale considerations raised above. Who selects educational objectives 
and who decides what Innovations will be adopted or rejected? When of- 
ficials of a national government select objectives and when accountability 
at that level Is under consideration, then the gross national product Is 
the most convenient Indicator of attainment. In the past, the Implicit goal 
of education, so far as national leaders and the GNP are concerned, has 
been to Increase the amount of time spent In school per student, to Increase 
the number of students benefiting from school, and as well to Increase the 
proportion of students who attain graduation status In the higher levels of 
education. Economists characteristically deal with these variables and they 
have reasonable evidence to show their relationship to the GNP (Denison, 1964, 
pg. 23). 



The present view of the Trent Valley Centre staff Is that GNP 
Is an Inadequate measure of Improvement within a society (the argument Is 
basically that proposed by Galbraith, 1969, pg. 124). Furthermore, 
economists In education and educational administrators should accommodate 
their views to the position taken by Oenlson (1962, pg.67), 

"It (GNP) can deal only with changes In the amount of formal education 
received by members of the labour force. It cannot take Into account 
changes (presumably Improvements) In the quality of the day's 
schooling." 

Denison also asks, 

"can these schools teach as much In less time or at less cost through 
attention to the critical objectives being achieved, than through 
Innovative means to maximize student attainment of the objectives?" 
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The Trent Valley Centre has bean Influenced by such considerations and Its 
attention to precise assessment of student performance on educational ob- 
jectives Is an essential Ingredient In the TVC Program. 

When national leaders have the responsibility for setting educational 
goals, either explicitly or Impllclty, It seem that the accountability Issue 
Is relatively clear and that attention to specific student performance ob- 
jectives will probably lead educational researchers and educational 
administrators In the right direction. However, there are problems with 
educational goals for a large unit of Jurisdiction, whether supematlonal, 
national or subnatlonal. Even the act of setting goals over such a large 
unit of Jurisdiction may be df afunctional. Dewey has commented on the effects 
of nationally or externally Imposed goals In Democracy and Education (Dewey, 

1969, pg. 0)* 

t 

’’The vice of externally Imposed ends has deep roots. Teachers receive 
them from superior authorities; « v hese authorities accept them from 
what Is current In the community. The teachers Impose them on 
children. As a first consequence, the Intelligence of the teacher 
Is not free; It Is confined to receiving the alms laid down from above. 
Too rarely Is the Individual teacher so free from the dictation of 
authoritative supervisor, textbook on methods, prescribed course 
of study, etc., that he can let his mind come to close quarters 
with the pupll*s mind and the subject matter. This distrust of 
the teacher* s experience Is then reflected In lack of confidence 
In the responses of pupils. The letter receive their alms through 
a double or treble external Imposition, and are constantly confused 
by the conflict between the alee which ere natural to their own 
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experience at the time and those In which they are taught to 
acquiesce. Until the democratic criterion of the Intrinsic 
significance of every growing experience Is recognized, we shall 
be intellectually confused by the demand for adaptation to 
external alms.” 

The Trent Valley Centre staff recognizes the basic significance 
of Dewey* s comment and accepts some responsibility for influencing school 
people to consider alms of education that are rooted In the students and 
the people of the school community. By accepting such a view It is not 
necessary to exclude alms of education for larger Jurisdictional units: In 

fact, the need for such alms Is obvious. Such alms must be sufficiently 
general that Individual students and teachers may be free to Identify and 
pursue specific local and Individual goals. 

A. N. Whitehead (1956) provides even more precise guidelines for 
the Identification of the person or persons who should have responsibility 
for Identifying educational objectives. Whitehead says, 

w E'it the first requisite for educational reform Is the school as a 
unit with Its approved curriculum based on Its own needs and evolved 
by Its own staff. When I say that the school Is the educational unit, 
I mean exactly what I say, no larger unit no smaller unit." 

(Whitehead, 1956, pg. 21). 

If Dewey and Whitehead have provided good advice on this Issue, then the 
"who" question raised earlier must be answered In a way that provides a 
high degree of school autonomy and Individual freedom. 

Independently of the issue of compatibility between school goals 

and national goals, there Is good reason to study carefully school goals, 
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and to accurately assess the degree of attainment of these goals and their 
specific subgoals or student performance objectives. This Is the business 
of evaluation and, furthermore, student performance data can serve as a 
reasonable starting point for the consideration of school change and the 
Introduction of Innovations. The Identification of Innovative programs 
that are compatible with the students’ needs and the community’s ability 
to pay the cost Is a sophisticated business that sometimes leads to the 
adoption of rigid packaged programs and at other times to the preparation 
of school-mad©, school-unique programs. Whatever the Innovation, the 
adopt-adapt-reject decision must be related to precise evaluation data, 
and In cases where positive benefits ore the clear result of on Innovative 
program, the Implementation of an effective diffusion program can magnify 
and multiply the benefits to a larger population of students. 

The Trent Valley Centre working model consists of both components 
and stages. The components, seven In number, are features found to be 
critical to change which appear In one or more of the stages of change. 

The stages are seven temporally sequenced points on a continuum beginning 
with a decision to begin working toward change and ending at the stage of 
field trial, from which point recycling through some earlier stages Is 
still likely to occur. The major components of the model Include: (a) a 
climate for change; (b) academic-practitioner Interaction; (c) roles for 
evaluation; (d) program development strategies; (e) Interschool cooperation 
(f) county wide communication networks; and (g) teacher responsibility f^r 
change. Stages In the model Include: (a) agreement to begin; (b) establish- 

ment of an organization; (c) selection of problems and goals; (d) study of 






available solutions; (e) pilot trials; (f) adopt, adapt, reject decisions; 
(g) field trial. The model may be thus conceptualized as a components X 
stages matrix (Figure 1) with a total of 49 potential cells some of which 
are obviously Important, some empty and many the significance of which has 
yet to be determined. Those components and stages that relate to evaluation 
are elaborated In some detail whereas the remainder are given only the 
amount of attention necessary to provide a meaningful overview of the 
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Figure I A matrix showing possible Interactions of components and 

stages In a school change model. 
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Cl I mate for Change 

The climate for change Is an elusive component to analyze, 
primarily because of the variety of forms It takes In different school 
contexts and the consequent difficulty In Identifying common underlying 
features. Classification of the type of change referred to, and Illustration 
of several situations where such a climate pervades, probably yields the best 
possible understanding of this component. 
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Types of change have been classified by many authors (Maguire, 1970) 
using a variety of criteria, one of the most powerful being the relationship 
between a change agent and client system (Bennls, 1966). All of these 
classtflcatlife however. Include a category labelled ’’planned change” 

and the variety definitions offered, for such change suggests that It Is 
congruent with the change being discussed here. Chin (Maguire, 1970) sug- 
gests that such change Is ”• • • a deliberate and collaborative process 
Involving a change agent and client system that are brought together to 
solve a problem or to plan and attain an Improved state of functioning In 
the client system by utilizing and applying valid knowledge (p. 1 1 > • " In 
the present model such a definition needs to be tempered by some of the 
characteristics of what Bennls describes as "technocratic change." This 
type of change relies on the client’s (teacher’s) definition of his problem, 
the agent’s knowledge of strategies leading to solution and the collection 
and Interpretation of data to facilitate and validate that solution. Such 
planned technocratic change Is endorsed as a systematic procedure for 
efficiently carrying out rationally preconceived alterations In the 
educational enterprise. Because such alterations In the curriculum area, 
as well as many other areas, are continuous, the roles of the client and 
agent. In this model, are not clearly distinct. The client must be given 
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the opportunity. In his Initial encounters with the agent, to acquire 
enough of the agent’s motivations and skills so that agent and client ere 
eventually dual roles played largely by the same personnel with decreasing 
external Inputs* Such welding of client-agent roles Into the person of 
the educational practitioner Is designed to overcome serious criticisms 
levelled by Herzog (Maguire, 1970) at typical approaches to the concept 
of planned change. These criticisms Include (a) viewing schools as objects 
to be manipulated and (b) failure to recognize that most people are engaged 
In activities because they see value In those activities not because they 
are resistant to change. A third criticism - that planned change Is too 
often naively profession-centred Is compensated for In the TVC model by 
systematic Involvement of the community In school goal specification. 

While the type of change (planned-technocratic) has now been 
established. It Is still not clear what the term '’climate for change" 
really means. Planned-technocratic change Implies a process of conscious, 
systematic and scientific response to rationally Identified educational 
needs or goals. Such a process requires large resource allocations on the 
part of those Involved and, hence, a clear understanding and substantial 
commitment to the need for change on the part of practitioners engaged In 
planning the change. The climate for change Is the behavioural ly represented 
Indicators of such understanding and commitment. Such a climate seems to 
exist (a) when teachers wish to meet with parent groups to cooperatively 
engage In school goal setting, (b) when teachers meeting as a group openly 
discuss their real problems and cooperatively plan toward solutions, (c) when 
a principal Is willing to assume a facllltatlve rather than directive role 
with teachers, (d) when a school staff desires to rearrange timetables to 
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free blocks of cooperative planning time, (e) when a teacher feels secure 
enough to oppose, on rational grounds, changes suggested by the principal, 

A| I of these Indicators of what Is meant by a climate for change are found 
In the schools In which the change model Is developing and many more could 
be cited. Suffice It to state that the climate for change Is a pre-condition 
to planned change or even planning for change, and without It subsequent 
actions would probably be largely Ineffective. It Is, In fact, the failure 
to establish such a general climate for change that has doomed many curriculum 
development projects to failure before they have begun. 

The ways In which a climate for change Is established vary greatly 
across schools, but In most Instances the principal Is responsible for Its 
Initiation. Several eppendums to this statement should be noted Immediately, 
First, while the principal Is a centre of communication (see page 29 ), the 
teacher has ultimate responsibility for effecting classroom change (see page 29 ) 
and hence the Initiation of change by the principal must be as a stimulant to 
the teacher^ assumption of responsibility for change. Such Initiation cannot 
be forced nor should It reduce the teacher*s Important decision-making 
responsibilities, but only make clearer how the principal can be used to 
facilitate the changes envisioned by the teacher. Most Important, the 
teacher must be made aware that his actions are endorsed and supported by 
the principal. Second, It would be misleading to suggest that the methods 
used by principals In the project so far, to Initiate change, have much or 
any general Izabl I Ity beyond the schools In which they were used. Principals 
who encountered difficulty In stimulating their staffs to assume responsibility 
for change when the TVC Program began are, almost to a man, still having a 
great deal of trouble, In spite of the benefit of consultation with fellow 
project principals having more success In this regard. The establishment of 
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a climate for change Is a highly complex endeavour Involving an optimum 
mixture of principal support, staff readiness, group dynamics, problem 
visibility, community characteristics and a host of other, less readily 
Identified variables. When the right mixture exists, a possibly false 
Issue such as low school scores on a standardized test In mathematics 
can be sufficient to elicit a commitment to change which will carry those 
Involved through the labour of all seven of the model’s stages of change. 

A more authoritative evaluation of school programs using more valid data 
Is clearly a preferred means of beginning. 

Academic-Prac t itioner Interaction 

The academic-practitioner Interaction Issue has remained virtually 
unresolved over most of the history of education. As a consequence, the 
number of solutions generated by educational researchers to problems of 
concern to practitioners Is woefully small. Several reasons for this state 
are readily evident. Educational researchers are too often engaged In 
finding solutions to problems Identified by themselves and Irrelevent to 
school people. No matter how significant the solutions thus generated, 
their Impact will be minimized by the lack of need for such solution. Re- 
search activities, to be productive In school contexts, must be organized 
around Issues considered critical by practitioners. This means (a) creating 
two-way communication links between the researcher and practitioner and 
(b) utilization of expert manpower to solve the emerging Issues even If 
that means engaging researchers In projects which are not of natural concern 
to them. The Idea of the researcher, using his scientific skills In pursuit 
of knowledge vital to the Interest of others and not necessarily himself. Is 
largely foreign and probably distasteful to many educational researchers. 

What Is being suggested Is the adoption of a model for client-agent Interaction 
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which more nearly approximates the Industrial rather than the university 
model with Its record of brilliant yet often disjointed and Ineffectual 
achievement. Perhaps the best of both worlds Is possible with the proper 
matching of a large pool of expert manpower resources with problems judged 
relevant by external , school sources. 

Even when academics have dealt with school relevant problems, 
the strategies for operationalizing change, as often as not, have been 
Impotent. Responsibility for this condition rests evenly with academics 
and practitioners. Academics hove typically adopted dysfunctional tactics 
characterized by a large proportion of discussion of problems and solutions 
at a general level, and a very small proportion of expended effort In the 
task of change Implementation. It Is erroneous to suggest that the 
generation of solutions and their Implementation are separate activities. 

In fact, only through the labour of attempted Implementation does a 
solution acquire that ultimate richness which qualifies It as a worthy 
achievement. Part of this unfortunate situation must be attributed to the 
Implied wishes of practitioners, however. An Invitation Issued to an academic 
"expert" to speak at a professional development day on some related occasion 
Is one of the best possible ways of confusing an Issue and building In a 
resistance to change on the part of a majority of those who attend such an 
occasion. Change will only occur when an organization Is established to 
facilitate communication on a continuing basis. If Inspirational speeches 
were In fact effective, every school would be a model of planned change 
since all staffs have been subjected to such Inspiration many times. The 
lecture strategy, therefore, may serve to give the appearance of change 
activity and avoid the potential trauma associated with tho realities of 
change, 
o 
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Roles For Evaluation 

Scrlven (1967) has distinguished between goals and roles for 
evaluation. Although there Is some disagreement, (Stake, 1967), the goal 
for evaluation In this model Is singular, as Scrlven suggests, that being 
to Judge the merit or worth of an educational variable. Ths most con- 
troversial Issue here, In fact, Is not whether there are other goals for 
evaluation but who will be the Judge. Consistent with the concept of 
planned-technocratic change, discussed above. In which there Is gradual 
welding of client-agent roles, and contrary to the expressed preference 
of Scrlven (1967), the Judge In this model Is the cllent-practltloner. 

The Initial change agent assumes consultative responsibilities with regard 
to data analysis, Interpretation and research design, where they are 
Important, but the adopt-adapt-reject decisions are exclusively In the 
domain of the practitioner who must ultimately Implement the decision and 
be held accountable for Its consequences. 

The roles of evaluation are many Including prediction, selection, 
national assessment, diagnosis, curriculum assessment and many others. 

These various roles have been classified as either formative or summatlve 
depending on the uses made of the resulting data (Scrlven, 1967). Summatlve 
evaluation provides Information to enable a potential adopter of an educational 
process or product to determine the relative costs, advantages and dis- 
advantages of an Innovation prior to making a commitment. When packaged 
programs are selected by schools, this Is the use to which evaluative data 
are put. Formative evaluation, on the other hand, proposes to supply 
Information to the developer to assist In the further refinement of process 
or product. As a result, formative data may often have to be very detailed 
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and particularly multl-dlmenslonal In order to be useful. Furthermore, 
they must be available when they will be useful to the developer. Where 
the teacher Is also curriculum developer, data from the previous year’s 
program must be available before beginning the new year’s program, and dat/j 
from each segment of the program must be available as the subsequent ap- 
plication of the program segment Is planned. 

Within this formative role, although not exclusively, evaluation 
serves three distinct functions In the change process. These three functions 
are the promotion, facilitation and validation of change. Evaluation data 
often function to promote change by stimulating attention to problems with 
the status quo and providing a basis for making decisions about program 
adequacy with regard to current educational objectives* A decision to 
evaluate present curricula also leads through a process of goal clarification 
and refinement which virtually guarantees that the program finally evaluated 
will not be the original program. This Is one of the effects of experimenter 
Intervention that Is to be avoided at all costs In controlled research, but 
on the other hand It Is to be greatly encouraged In the process of planned 
educational change. Evaluation has thus begun to facilitate change and can 
do so further by enabling developers to Identify program objectives being 
achieved least well* as a focus for the Initiation of systematic curriculum 
development. Such Identification of Initial focus has the advantages of: 

(a) diagnosing weaknesses teachers may be able to Improve on 
Immediately In the classroom; 

(b) scaling down the size of the curriculum development task by 
avoiding work on objectives already being well achieved. 

Both of these features are especially attractive when teachers are also 
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developers since they are likely to feel the press of dally classroom needs, 
and have only enough time to work on the most urgent curriculum problems. 

The third function of evaluation In change, that of validation, 
speaks directly to much of the available change literature which appears 
to Imply that change Is to be valued In Its own right without regard to 
the consequences of that change. The TVC model diverges most severely with 
the literature In taking considerable pains to assess the effectiveness of 
change and In providing opportunities to adapt or reject Ineffective changes. 
Many dimensions of Innovative products, chosen for or developed In the project, 
have been measured to date. They Include student, teacher, teacher aide and 
parent attitudes, description of treatments, student performance and the 
relationship of student performance with baseline descriptions Including IQ, 
socioeconomic status and standardized achievement scores (see Chapter 6, 
Educational Changeand Innovation: A Case Study). Particular attention has 
been paid to student performance data on the assumption that an Innovation 
to be worthwhile must result In student achievement that Is higher, different 
and more valuable, or less costly than In the previous program. (An unlikely 
but possible exception to this would be an Innovation which had a positive 
effect on attitudes and produced the same amount and type of student per- 
formance as the previous program at the same cost.) 




The assessment of student performance has moved the focus of 
measurement within the model away from classical norm-referenced measurement 
techniques Into criterion-referenced measurement. This shift has taken place 
not because of a profound disagreement with classical measurement methods 
but because of a difference of purpose. Classical measurement methods are 
designed largely for selection and prediction. Because of this, test 
development procedures tend toward making such Instruments uni dimensional. 
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Assessments, using such techniques, Indlcete where a testae stands In 
relation to other testees with respect to some Identified measure or some 
clearly prescribed .summat Ion of measures. However, the threefold purposes 
of evaluation In the TVC model require Information of a very different sort. 

Within the TVC model the student variables being measured may be 
heterogeneous rather than homogeneous, and each component part may require 
separate assessment. This appears to be the case In practice where the dual 
purposes of formative evaluation as used In the model Include both diagnosis 
and curriculum evaluation. In each case, student achievement of specific 
educational objectives Is the Information being sought and selection and 
prediction decisions do not enter. In spite of the basically different 
functions, the same pool of test Items may serve both classical and 
cr Iter Ion- referenced measurement purposes. The differences In purpose 
Will dictate differences In procedures for Item selection from the common 

pool. 

The shift In enphasls from norm-referenced to criterion-referenced 
data has some additional positive features that were not taken Into account 
during the Initial discussions, that have led to the shift. A renewed 
emphasis on the Individual rather than society, and a new fascination with 
the negative effects of competition and aggression seem evident not only In 
current literature but also In official government policy statements such as 
the Commission on Post Secondary Education ( 1972 ). Criterion-referenced 
data do not eliminate competition, nor do they Inevitably lead to a focus 
on the Individual rather than his group. However, such data do not depend 
on, nor do they emphasize quantitative discriminations among students. Also 
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they more readily lend themselves to heterogeneity In goals among Individuals 
and to the use of data by the Individual who Is measured. 

Process evaluation Is considered to be as Important as product 
evaluation but It Is more difficult to develop an appropriate methodology 
for It. Several promising plans have recently been Initiated In cooperation 
with OISE sociologists Michael Fullan and Glenn Eastabrook. These plans 
Involve Initial data collection of school characteristics and modes of 
operation related to Innovativeness. Subsequent Intervention programs will 
be undertaken, when project schools desire It, to alter, where possible, 
characteristics which detract from optimum change activity. To date, the 
collective processes Imbedded In the change model appear to be reasonably 
affective, but the process evaluation of the next two years will be necessary 
to Identify specific strengths and weaknesses of these processes. 

Student achievement In the project curricula have been, or will 
be In the near future, assessed employing one or more of three experimental 
designs. Using Campbell and Stanley's (1963) terminology these designs 
Include (A) the ’’one-group pretest-posttest" pre-experl mental design, 

(B) the "nonequ I valent control group" quasi -experimental design and 

(C) an extremely useful adaptation of the pre-experlmental "static group 
comparison." 

This yaar the "one group pretest-posttest" pre-experlmental design 
was used to evaluate student achievement In all of the project components. 
This design yields Information limited by a number of extraneous variables 
that can Jeopardize Internal validity. Those variables Include history, 
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maturation and the affects of testing. The effects of history and maturation 
will be greatest In the case of two packaged programs since a greater time 
elapsed between pre and posttesting. Results from teacher-built programs, 
because of shorter elapsed time are less Influenced by history and 
maturation, but more affected by the tendency of students to do better 
on the second administration of a test or alternate form of that test. As 
a result of the possible Inflationary effect of these threats there Is 
Interpretation of results. Nevertheless, as already discussed, because 
two teacher-built curricula are In the formative stages of development, 
the Inadequacies of this design are to be preferred to a more rigorous 
design which might have discouraged further program development - the 
promotion of which Is a major role for evaluation In this context. While 
Inadequate resources are a major reason for employing this same design 
with two highly developed packaged curricula, the benefits of the design 
remain compelling. These have to do with the freedom teachers are given. 

In a non-comparative setting to develop not the already fixed program 
components, but the necessarily school specific techniques required to 
make the program opttmutty successful. Further, the pressure of 
Implementing a new curriculum In the first year could, under some cir- 
cumstances, be added to by the knowledge of comparison. 

The quasi -expert mental "nonequt valent control group" design was 
used with pre and posttesting In one Instance this year and Its use will be 
expanded next yuar to Include four programs thus far evaluated. Socio- 
economic status (as well as school size) was used as a basis for control 
because of both Its potency In relation to achievement variables (Ireton, 
1970) and the availability of such data without large resource expenditures. 
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Depending, to some extent, upon the equivalence of groups on pretest 
scores this design controls the main threats to Internal validity of 
history, maturation, testing and Instrumentation since these factors 
should Influence both control and experimental groups similarly. Re- 
gression effects are not likely to occur since there Is no need to be 
concerned with extreme groups. 

An adaptation of the "static group comparison" design has been 
made which makes the design Inexpensive, convenient and practical to employ, 
as well as yielding unusually dependable comparative data for evaluation 
purposes. Typically, this design compares the p03ttest results of a group 
which has experienced a treatment with one which has not for the purpose 
of establishing the effect of the treatment. The comparison group consists 
of the pupils In the same grade, same school with the same teacher the prior 
year. For example, an Innovative kindergarten program being Introduced In 
September 1971 Is to be evaluated with this design, so In May 1971 the grade 
1 children (last year's kindergarten class) were posttested and these re- 
sults compared with the May 1972 results of the class taking the Innovative 
program. The assumption of pretest group equivalence cannot be accepted 
as readily as would be the case had comparison and control subjects been 
assigned to groups randomly. Yet history Is virtually the only confounding 
variable to a highly effective group match given the relatively stable 
population from which the subjects are drawn. In many cases the children 
even have the same parents. Certainly, group measures of SES, IQ, and 
general Information as well as measures of school environment and community 
environment are almost the same for both groups. 

This design has a number of Important advantages for development 
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work, the major one being Its economy. Comparison groups are always 
dltflcult to find and non- Innovating schools are understandably reluctant 
to participate In studies for which they perceive little payoff to themselves. 
Using this design, both control and treatment groups can be drawn from the 
Innovating school. In order to provide both groups using randomization 
techniques, half of the target population must forgo the Innovative treat- 
ment In order to serve as comparisons, or the treatment can be offered to 
half of the target population for the first half of the year and the 
remainder of the target population In the second half of the year. This 
latter arrangement necessitates Innovating during only half of the school 
year. None of these requisites for randomization are convenient for school 
people and none are necessary with the design modification suggested. The 
main need for caution with the proposed design concerns the degree of In- 
stability of the school population and environment. Rapid changes can 
Invalidate the design and thus It becomes Infeasible In spite of Its 
apparent strengths. 

Several other l,ssiies related to the operationalizing of ex- 
perimental evaluation design are appropriately dealt with here. The first 
concerns procedures for data collection, specifically the collection of 
criterion-referenced data. As discussed earlier, collection of such data 
Ideally Include comprehensive assessment of student achievement of all 
program objectives, resulting In many test Items. In most school settings 
the amount of time required to test all students on all Items becomes 
prohibitive. Three solutions to this problem are most readily apparent 
and two have been tried In the TVC Program, to date. One Is to Incorporate 
the testing Into the ongoing program using the Items for teacher-diagnostic 
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as well as overall evaluative purposes. A second solution Involves random 
assessment of Item samples to participating students. The third solution 
depends on an emerging analytical technique that requires a minimum of Items 
to focus on a students particular domain of learning. The first solution 
has the advantage of providing Immediate feedback to the teacher and Is 
difficult to surpass as a formative evaluation strategy when the teacher 
Is also the curriculum developer. The disadvantages, however. Include 
Inconsistent testing procedures since there are many testers, as well as 
placing the responsibility for carrying out the testing schedule largely 
on the shoulders of teachers. When the data are to be used for summatlve 
purposes, experience Indicates that In spite of relatively open com- 
munication channels, the necessity of carrying out the testing schedule 
and procedures exactly as planned may not be fully understood. This 
severely Jeopardizes the reliability of data. 

A more satisfactory compromise now seems to be the provision of 
diagnostic test Items for use In the formative mode as well as a formal 
testing program carried out by R and D personnel. In order to undertake 
the latter, techniques have to be found to both dramatical ly reduce testing 
time while at the same time assessing most measurable program objectives. 

One method of doing this, that has been satisfactory In the TVC experience. 

Is to randomly divide the total Item pool Into a number of sub-tests, each 
requiring about the same period of time to complete, and then randomly 
assigning sub-tests to students within a given class. This strategy results 
In a mean score for each Item, for each class, rather than a score for each 
student. Such a result Is particularly attuned to the purposes of criterion- 
referenced measurement since the adequacy of the program In achieving each 
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of Its objectives becomes the focus of evaluation rather than the adequacy of 
the student. 

The third solution Is under study by the TVC during Its development 
by OISE staff (see Olivier 1971 ). Through the careful preparation of de- 
pendency trees, Olivier has been able to select test Items that In themselves 
provide Information concerning the many dependent Items from lower positions 
on the dependency tree. At the present time the Olivier system Is operational 
In a computerized remedial mathematics program, but a hard copy product of 
the program can be used effectively without access to a computer facility* 

Frequently criterion-referenced measures consist of large numbers 
of performance Items which place heavy demands on the time of the evaluator 
If formal testing procedures are to be carried out. In the project the time 
of testers has been severely limited resulting In a search for adequate 
methods of expanding evaluation without a reduction In the quality of the 
data. In one program now under evaluation (to be reported next year) a 
highly selected group of volunteer parents wore trained In the administration 
of Individual performance test Items In science. From the R and 0 evaluation 
point of view, the data they collected were highly reliable due to the ex- 
tensive training and monitoring of testing procedures. In fact, because 
parents were chosen who had children the same age as the subjects being 
tested, the rapport of tester and testae often surpassed the rapport a 
professional tester would be capable of establishing* From the point of view 
of the principal Implementing the program that was being evaluated, the use 
of parents as testers had several added positive features* The parents, 
having become thoroughly familiar with the school operation, In general, and 
the new program. In particular, acted as goodwill ambassadors to the com- 

© munlty for the school. Exposure to the school under structured conditions 

ERJC 





24 - 



also added considerable Impetus to this volunteer parent program. 

Cu rriculu m Dev elopment Strategies 

In spite of the Importance of processes for change, the criterion 

against which they must be Judged Is the outcome or product of those pro- 
cesses. It would be difficult, however, to defend an absolute distinction 
between process and product since a product like "student achievement," as 
It can be measured. Is only a static and, therefore, artificial record of 
continuous learning and performance. An operational distinction can be made 
where classroom treatments such as IPI or Mathematics A ere defined as pro- 
ducts, activities preparatory to such treatments such as teacher planning 
and In-service training are processes, and student achievement Is the out- 
come criterion against which both product and process are Judged. When the 
product Is defined as classroom treatment, the limitations of both product 
and process evaluation become Immediately evident. Such evaluation, con- 
cerned with student achievement. Is an assessment of unique, non— repeatable 
treatments when those treatments are each considered as a unit. There are 
critical components of those units that are repeatable, however, and the 
curriculum development processes or strategies In this model focus on these 
repeatable components. 

The strategies employed In program development In the model Include 
four dimensions (Figure II), these being the Identification of educational 
objectives, the gathering or generation of materials to be used In achieving 
those objectives, the choice of Instructional techniques for manipulating 
those materials In an effective manner, and evaluation of the achievement 
of objectives, materials and techniques. 

The sorts of formative and summatlve evaluation required and the 
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selection strategies exercised on materials and techniques necessitates the 




Figure II - - - Components within curriculum development strategy* 



specification of objectives to be In operational, behavioural or student 
performance terms. While many arguments have been raised against such 
specification, most are patently Invalid (Lelthwood, Russell, In press; 

POpham, 1970). The position being forwarded really Is no more complex than 
Indicating ones purposes so that one can determine how to go about achieving 
those purposes and assess when or If those purposes are achieved. 

The order In which the components appear In the curriculum 
development process vary greatly with the needs and characteristics of the 
developers. In many Instances the shortest route to total curriculum develop 
ment task ’’completion" (the task Is really never done) may begin with the 
accumulation of materials component. Beginning with selection of materials 
Is useful since It leads to Innovations In the classroom very quickly and 
generates the need tor move concretely to Issues of objectives as a basis for 
decision-making about selection of materials. Beginning with evaluation has 
the advantage of diagnosing areas of greatest present weakness and treating 
those areas of greatest need first. It necessarily Involves beginning where 
one Is at present and moving from there - - a very sound policy. Beginning 
with the objectives component Is logically appropriate but objectives 
specification can be a long and difficult task requiring considerable 
patience before the Impact of such work becomes visible In the classroom. 



One of the most promising way* of reducing the size of objectives 
speclflcetton task Is through the use of objectives pools like the In- 
structional Objectives Exchange (Popham, 1971) or, even better, objectives 
and Items pools (Horn and Russell, 1971? Lelthwood and Russell, 1971). 

Such pools enable teachers to select student performance objectives, which 
they feel are appropriate for their curricula, without the need of 
generating such objectives on their own. When the objectives have ac- 
companying Items, two of the four components of curriculum development, 
objectives specification and evaluation, are greatly facilitated. The 
curriculum developer’s major concerns can then be focussed on objectives 
selection and the materials and techniques components of curriculum develop- 
ment. It might also be possible for materlals'developers to relate closely 
to such pools and help create teacher resource centres where, not only 
objectives and test Items, but associated materials were catalogued 
(Lelthwood, 1971). This Is not to suggest that the same materials cannot 
be used to achieve many objectives but some are obviously Inappropriate 
and a few extremely useful for Instructional purposes related to a specific 

educational objective. 

The arrows In Figure II Indicate that curriculum development can 
begin with any component and move In a number of directions. It Is not 
possible, however, to arrive at the evaluation component without going 
through the specification of objectives first. Having arrived at evaluation 
however, the resultant data potentially feedback Into all four components 
for further revision and refinement. Suppose, for example, that the results 
of evaluation Indicate no student performance gain on a given objective 



after exposure to the curriculum treatment. Fifteen possibilities 
potentially are available to account for such lack of program In- 
effectiveness. These refer to decisions about each of the components In 
Figure II considered separately as well as all possible combinations of 
such components Including: 

1. The objective was Inappropriate, unachievable or otherwise poorly 
selected for the program and requires revision or elimination; 

2. The program materials designed to achieve that objective are Ineffective 
and require revision or change; 

3. Instructional techniques need review; 

4. Items used to measure objectives achievement are Invalid; 

5. to 10. Problems with double combinations of components Including I and 2, 

I and 3, I and 4, 2 and 4, 3 and 4; 

II. to 14. Problems with triple combinations of components Including I, 2 and 
3, I, 2 and 4, I, 3 and 4, 2, 3 and 4; 

15. Problems with all 4 components of the curriculum development process. 

Because each of the four components requires Its own set of skills 
to carry out adequate development, and because of the large number of possible 
decision points (15) requiring data of some sort# two observations seem 
notable here. First, the complexity of the curriculum development process, 
considered without reference to a larger framework of school change, has 
been grossly underestimated by school people generally, and perhaps by 
academics as well. Second, the availability of multl-dlmenslonal evaluation 
data Is vital. Both of these observations seem, on the surface, to suggest 
that asking the busy teacher to be a part of curriculum development Is very 
unrealistic. On the contrary, expecting anyone but the teacher-deve leper 
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to have access to the range of primary, secondary, objective and subjective, 
sometimes Impressionistic, Informally gathered data necessary to cope with 
the 15 decision points outlined Is even more unrealistic. The task Is 
slnply beyond the scope, resources and technology of the professional 
curriculum developer and evaluator. What seems to be most realistic Is 
substantial support for the teacher In his role both through extensive 
In-service training In a meaningful context, and facultative and consultative 
personnel and agencies prepared to act on needs Identified by the teacher. 



Teacher Resoonslbl II tv 

The rationale for primary teacher Involvement In curriculum 
development and school change, generally, has been foreshadowed In the 
previous discussion of curriculum development and welding the roles of the 
change agent and client Into the person of the client. The need for such 
responsibility and Identification of the school as the critical educational 
unit have been attributed In part to Dewey and Whitehead In the opening 
paragraphs of this paper. Both philosophers knew that If educational 
objectives are to be Implemented In classrooms they must be the schools 

i 

own objectives. When this Is not the case and objectives are Imposed on 
the teacher, a conflict typically arises between dormant explicit ob- 
jectives and very active Implicit objectives with Implicit objectives 
most frequently winning. This consequent suppresilon of operational 
classroom objectives can only be detrimental to systematic curriculum 
development, communication and student achievement. 




Teacher responsibility for curriculum, however, does not Imply 
that superintendents, subject-matter specialists, consultants and trained 
curriculum developers cannot lighten the teachers* load greatly without 
violating the principle of teacher responsibility. What Is suggested 
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Is that teachers should be encouraged to Identify the needs they see In 
their classrooms, specify (at some level) the objectives they have for 
their students, feed other persons when necessary with Information to guide 
the development of materials and strategies that will help meet those 
Identified needs and critically evaluate materials and strategies In light 
of their objectives. It should be the teachers* prerogative to make the 
final adopt-adapt-reject decision (within the financial limitations Imposed 
on administration) because It Is the teacher who Is ultimately held ac- 
countable for the performance of the students subjected to the curriculum. 

Communication networks and Inter-school cooperation 

Two networks for communication have been elaborated In the model* 

The first Is a communication network which relates educatlona-l personnel 
by constituent position to one another where the school principal Is the 
hub of communication with respect to change and Innovation. The central role 
of the principal In this network highlights his function as change agent having 
direct communication access to senior administration, teachers, students. 




(AFT) 

Figure III The Central Position Of The Principal In Educational 

Communication 
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parents and outside agencies, although no direct contact with elected 
trustees In most cases* He Is, therefore. In a position to Initiate and 
facilitate change In his school and the school, as already suggested. Is 
the primary educational unit* 



/ 




The second communication network functions as a mechanism through 
which the principal and sometimes staff representatives performs many of 
their facilitating activities. This network links school principals 
formally Involved In the Innovative process through cooperative groupings 
of various sizes depending upon purpose* Those types of groupings are 
Imbedded In this network. The first of these Is labelled a "Joint" group 
and consists of all Innovative school principals In a county who wish to 
be a part of the change model* This group's function Is to provide (a) a 
forum for general Issues of common concern and (b) simple Information relay. 




Special 

Interest 

Groups 



Figure IV - - - A School Interaction Network To Facilitate Cooperative Change. 
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A second group, the ,, llel8on ,, group coordinates and facilitates the project 
work of the participating schools. More sub-groups are designed to deal 
with specific, substantive currlculun and other change Issues. The group 
structure provides (a) the benefit of mutual experience (b) the dissemination 
of useful Information (c) mutual support and (d) an efficient means whereby 
consultative assistance (e.g. Trent Valley Centre) can be mustered to deal 
with Important problems of common concern. Furthermore, new schools wishing 
to Join the project may do so at a relatively concrete level through a 
special Interest group, although Joining the project does carry with It the 
responsibility of studying change processes more generally. These new 
schools may be considered a special Interest group focussed on change and 
representatives from the liaison committee play a consultative role In that 
group. The groups last only as long as It takes to solve the specific 
problem they were created to work on. 

Some of the advantages of this organizational network are better 
appreciated when viewed In a broader perspective. An Interesting paper by 
E. 6. Bogue (1971) entitled "Disposable Organizations" provides such a 
perspective. Bogue contends that traditional organizational structures are 
often not capable of providing the fast acting response needed to deal 
effectively with contemporary change. Bureaucracies,' with hierarchical 
systems, devour the status quo and contribute to Inertia by reducing 
opportunity for change. One solution to this problem Is organizational 
decentralization but county reorganization In Ontario education was a move 
In quite the opposite direction. The special Interest groups In the network 
outlined above are what Toffler (Bogus, 1971) calls "throw away" of 
"disposable" organizations. Such organizations are (a) problem or Issue 
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centred rather than function centred; (b) temporary with a built In self- 
destruct mechanism activated upon problem resolution; (c) staffed so that 
authority of competence replaces the authority of position and role. This 
provides opportunity to utilize diverse specialists In a common venture; 

(d) able to short-circuit channels of communication rather than follow 
vertically structured paths. 

Bogue cautions, however, that "most persons need a degree of 
stability along with the challenge of change." Certainly the network being 
discussed has a relatively stable, although flexible Joint Group and the 
entire network operates cooperatively, rather than competitively within the 
more familiar educational administrative structure. There seems to be 
good reason, theoretically, therefore, for such a structure to be effective 
In facilitating change. 

Trent Valiev Centre Model Stages 

While the components of this model, as they have been discussed, 
are Important to consider In Isolation, many (although certainly not all) 
of their features are already well known and. Indeed, have been part of 
educational knowledge for many years. The major contribution to new 
knowledge that this model makes has to do with the ways In which the 
components Interact In a dynamic way to produce validated educational 
change. Educational literature abounds with treatise on objectives, 
evaluation and communication. The school as the educational unit and 
teacher responsibility for goal setting have been recognized many times 
and by many leaders, and yet few attempts have been made to Integrate 
these Ideas In a practical setting so that each component fits within a 
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lergor framework or strategy that will eventuate In the kind of change 
that Is really needed. 

Figure V Illustrates how the components of this model Interact, 
as conceptualized to date, through seven temporally sequenced stages from 
initial agreement to change to the stage of field trials of new programs. 
The latter two stages are as yet not clearly defined and they represent a 
focus of research for the next two years. It should be noted also that 
some of the elements and orders within each of the other stages will change 
as work progresses to refine the model. The work began. In fact, as a 
"model of" change and now most refinement Is In the direction of making It' 
a "model for" change. In the remainder of this paper the stage-based 
component Interactions wl 1 1 be discussed. 
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Figure "ST Stages leading to school change 
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Key to Figure “27 



1. Agreement to begin 




Meeting of representatives 
Meeting of pr & T representatives 
Meeting of i\ & D 
Meeting of consultants 
Meeting of other manpower groups 
Specific agreement to proceed 
Plan advisory committee 



2. Establ Ish organ! zatlon 




Select schools 

School level meetings, T, pr, + 

Meeting of pr, adm, R & D, consult. 

Pr study change process 
Select T within schools 

T study change process within school during released time 
Pr plans use of expert & volunteer manpower 
T make go or no go decision 



* Pt - parents 

* Pr - principal 

* R & D - Research & Development 

* T - teacher 

* St - student 

* Adm - administration 
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3. Select problems and goals 




T study general school goals 
Pr acts as faci I Itator 
Pr study behavioural goal specification 
T study behavioural goal specification 
School community Interacts on gen goals 
Pr study available solutions (consultants) 
T study available solutions (consultants) 
T generate or select Instr. goals 



4. Study available solutions 




T select inn. prog. 

T decide to dev. inn. prog. 

Pr seek authority to proceed 

T training + study 

T study materials available 

T select materials & organize 

T develop or select instructional objective 

T prepare auxiliary material 



5. Pilot trial 




Design pilot trial (time sequence) 
Pre-test 



Begin trials 
Formative eval. 
Adopt decision 
Adapt 
Reject 
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6. Adopt, adapt or reject 

Recycle prog, within school 
Report eval. data for prog. 
Adopt 
Reject 
Adapt 

Report summatlve eval. data 
Plan field trial 

7. Field trial 

Design for eval. 

Design for diffusion 
© Plan eval. of diffusion model 
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Stage 1: Agreements to Bag In 

This stage begins with a decision at the county level on the part 
of some group of practitioners (a) that their schools could be better and 
that they ore prepared to Investigate how such Improvement could be 
pursued* Having made this decision, the agreement to begin In a formal 
manner (f) necessitates gaining the cooperation of all effected groups 
Including principals, teachers, administrators, consultants and R and D 
persons who may be able to facilitate the desired change* This gaining 
of cooperation Is the purpose of meetings (b), (c), (d) and (e)* The 
result of these meetings and formal agreement to begin Is the planning 
of an advisory committee (g) which has a broad educational community base 
and Is designed as a sounding board on which the Innovators can test 
their ends and means before and during operationalization of these means 
and ends* Basic to decisions made at this stage are the concepts of 
teacher responsibility and the school as the critical educational unit* 

Five principals working toward their M.Ed. degrees were the original 
Initiators of change In the project from which the model Is derived* 

Stage 2; Estab 1 1 sh Organization 

The establishment of an organization for change begins with 
(a) a selection of schools. Two of the most Important Issues here are 
who does the selection and what are the selection criteria. Usually, 
the preferred situation Involves school self-selection on the basis of 
evinced Interest In change* When this Is the case the process moves 
directly to school level meetings (b) with and among teachers, principals 



and whoever else the school unit feels would be helpful - possibly R end D 
personnel and/or consultants. From this point two routes ore possible 
leading to the teachers* study of the change process In released time (f). 
One of these routes Involves the principal studying the change process first 
(d) as a means of determining techniques for Initiating an Interest In his 
teachers. This route would be followed when there was no strong Initial 
pressure from the teaching body to change. In such an Instance part of 
this study might lead him to plan for the use of expert and volunteer 
manpower (g) as a way of stimulating Interest. An alternate route, approp- 
riate when the prlnctpal Is ready to change and knows teachers are also 
ready Is to select the teachers who will begin (e) and Initiate their 
study of the change process perhaps In cooperation with him. Teacher 
selection again Is a critical Issue and self-selection Is vital where 
possible. The task of change Is a massive one, however, and for an entire, 
larger staff to be Involved simultaneously at the outset would present 
problems which might be Insurmountable. 

Where self-selection does not occur the route from (a) to <d) 
Involved (a) meetings of principals, administrators and R and D consultants 
(c) In order to facilitate selection. The gathering of soclometrlc data 
has been contemplated (but thus far not employed) for assisting In such 
selection. Two points need to be made clear here. First, the reaching 
of the "go or no-go" decision point with regard to change after a study 
of process Is most fundamentally the reaching of a point where the teachers 
decision takes priority. Second, there are two possible routes If the 
"no-go" decision Is made. One of these routes Is back to further study of 
the change process. The second route Is to drop out of Involvement In 
change at all, a difficult or Impossible thing to do. 
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Stage 3: Problem and Goal Selection 

The decision to proceed with change leads to a study of general 
school goals on the part of the teacher at a high level of generality 
Initially but at Increasingly specific levels as study proceeds. The end 
result of this stage Is either the generation and/or selection of specific 
program goals by the teachers or the selection of programs that speak to 
general teacher goals. In order to reach this stage , teachers must, and 
principals probably should, be Involved In studying the technology of goal 
statements In student performance terms and studying available programs 
that potentially achieve these teacher-generated objectives. The school 
community might also be Involved, as It has been In the project, at a general 
level. In order to ensure that the broad school goals reflected the goals of 
Its most relevant society. Depending on whether or not a ready-made 
program can be found or a new teacher-generated program Is to be developed, 
two routes Into stage 4 seem possible. 

Stage 4: Study Available Solutions 

The route from stage 3 (g) Is to stage 4 (a), the selection of 
the Innovative program and this route can be a reasonably swift one 
Involving principal search for authority to proceed and then teacher 
training and study of the program leading directly to a pilot trial at 
stage 5. If the Innovators enter stage 4 at point (b) they must examine 
available materials, select appropriate parts of them, organize these parts 
and write auxiliary materials (e), (f) and <h>. They must also design 
the Instructional techniques to be used with these materials <g>. They 
then are able to move Into stage 5 at the same level apparently as those 
who chose the other route. This equality of stages may be misleading. 
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however, since those who take the route (b) to (h) may need to recycle 
through stage 4 several times. If pre-selection criteria were adequate 
this Is less likely for those who chose the (a) to (d) route. 

Stage 5t Pilot Trial 

At this stage, the design of pilot trials using an appropriate 
evaluation design, pre-test, trial Initiation and the gathering of 
formative evaluation data are common steps, but alternate routes become 
available after this point. The decision as to route Is based on results 
of the formative evaluation and the possible routes Involve program 
adoption, adaption or rejection. If the data suggest adoption (a) the 
route Is directly Into stage 6. An adaption decision (f) may suggest 
recycling as little as simply beginning another trial with minor alterations 
or as much as beginning back In stage. 4. The rejection decision takes the 
Innovator back to program or goal selection In stage 3. 

Stage 6: Adopt, Adapt or Reject 

A decision to adopt at stage 5 leads to stage 6 recycling of the 
Innovative program within the Innovative school (a) and the gathering of 
additional evaluation data. On the basis of this data adopt, adapt or 
reject decisions are once again possible. An adopt decision leads to 
reporting to other Interested schools the results of evaluation and such 
data then are considered to be used In the summatlve mode. This report of 
data may lead to plans for program field trials In other schools who would 
be at point (f) In stage 3. An adapt decision, depending on the size of 
required adaption, may lead back to any point In sta<p4 and even back to (h) 
In stage 3, If problems are very fundamental. A reject decision might lead 
out of the change process. 
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Stage' 7: Field Trial 

The steps In stage 7 are not clear at this time and probably will 
not be until school programs reach this stage - a stage only one program Is 
now entering. It appears, however, that designs for evaluation and diffusion 
are Integral parts of this stage as Is a carefully planned evaluation of this 
diffusion model. 

Conclusion 

Evaluation as It has been elaborated In this paper Is an Integral 
part of a comprehensive program for school change. The Trent Valley Centre 
staff suggest that evaluation becomes most meaningful when It Is set In a 
curriculum development or school change activity that has goals, procedures, 
and people that are to be served by evaluation. What appears to be an over 
emphasis on the TVC model for change should be Interpreted as a further 
Indication of the great Importance that the TVC places on setting. 

The Trent Valley Centre program which Is elaborated as a diffusion 
model Is In one sense a summarization of the activities that have been under- 
way within the TVC and Its participating schools, over the past two and half 
years. In another sense It Is a description of a pattern of Interrelated 
tasks that other schools and school systems nay use to advantage, knowing 
that each part of the model has been tested out a number of times In the 
Peterborough setting, and as the so called model Is tried In other counties 
than Peterborough and as the Peterborough experience unfolds throughout the 
totality of county schools, there Is some reasonable prospect that a more 
refined and detal !ed model will emerge. 

Perhaps the most Important feature of the Trent Val ley Centre 
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program. Insofar as Its Impact on evaluation procedures Is concerned. Is 
the central role of the teacher as a curriculum decision maker. There Is 
no Intention here to Isolate the teacher as the sole decision-maker on 
curriculum matters, but on the other hand, by Identifying the classroom 
teacher as a primary decision maker, there Is a definite shift away from 
the common school practice of reserving major curriculum decisions for 
administrators and senior consultants. This downward decision-making shift 
has Implications for evaluation, curriculum development and school or- 
ganization. In the case of evaluation, the potential for school -specific 
and class-specific — and possibly even pupl I -specif Ic — objectives means 
that there will be a short feedback loop for data and rather than going to 
a county or a national office, they go back to the teacher and the student 
concerned. Decisions concerning school programs, class programs and 
Individual programs are based on the data specifically related to these 
programs. 

In cases where objectives are carefully elaborated for schools, 
classes and/or Individuals, It Is an easy step to generate criterion- 
referenced measures and another easy step to gather and report criterion- 
referenced data. These same test Items that ere used In a formative sense 
to make adopt-adapt-reject decisions at the classroom level can be used 
at a later point In time to make adopt-adapt-reject decisions about total 
programs or major sub-components of total programs. These same types of 
data used then In either the formative or summatlve sense to make adopt- 
adapt-reject decisions are In all cases gathered according to a design 
which Is some Instances Involves a parallel control group operating 
simultaneously with the experimental group, while In other and perhaps 
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In most Instances the control group Is a previous year’s cohort or class 
passing through the same grade with the same teacher In the same school, 
but under different program conditions. 

At one point In the paper the Issue of competitiveness was dis- 
cussed and some hope was offered that the use of criterion-referenced tests, 
as opposed to norm-referenced tests, could. In fact, reduce competitiveness 
and at the same time preserve Interest In the Individual for himself as 
opposed with his role within the group. The fact that norm-referenced 
tests result In a ranking of students along a continuum seems to encourage 
competition among Individuals, but on the other hand the criterion- 
referenced measures by virtue of their attention to specific goals could 
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under some circumstances set the stage for even more gamesmanship and 
competition. Perhaps those who want to reduce the amount of aggressiveness 
and competitiveness In students will have to find ways of achieving their 
end that are Independent of the testing procedures Introduced to assess 
student progress. 

It Is Important to suggest, as a last point, that the Trent Valley 
Centre program has avoided the Issue of Invalidation of data through the 
creation of false Incentives to Increase student scores. The reference here 
Is tc the short feedback loops that result In teachers having access to 
student performance data for the purpose of making program decisions within 
their classrooms and within their schools. County and province wide 
decisions pertaining to schools and teachers are not made on the basis of 
these same data, nor are school grants or dollar Incentives to teachers 
affected by these data. Any change In the Incentive system can serve to 
Invalidate the data and, as well. It may lead to the parrot-1 Ike learning 
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that students achieve readily under the pressure of a performance contract 
that has payoff totally unrelated to the Intrinsic value and the utility 
value of the learning Itself. 
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